User recognition and confirmation device and method, and central control system for vehicles using the same

ABSTRACT

A user recognition and confirmation device includes an image sensing unit, a face recognition unit, a display unit and an expression recognition unit. The image sensing unit captures a first image frame and a second image frame. The face recognition unit is configured to recognize a user ID according to the first image frame. The display unit is configured to show ID information of the user ID. The expression recognition unit is configured to confirm a user expression according to the second image frame and output a confirm signal. There is further provided a user recognition and confirmation method and a central control system for vehicles.

CROSS REFERENCE TO RELATED APPLICATION

This application claims the priority benefit of Taiwan PatentApplication Serial Number 101138995, filed on Oct. 22, 2012, the fulldisclosure of which is incorporated herein by reference.

BACKGROUND

1. Field of the Disclosure

This disclosure generally relates to an interactive system and, moreparticularly, to a user recognition and confirmation device and method,and a central control system for vehicles using the same.

2. Description of the Related Art

Presently the biological recognition technology has been widely appliedto the ID recognition. Generally biological information includes theinformation of, for example, face, voice print, finger print and pupil,wherein in the face recognition as a user does not need to take anycorresponding action, it is the most convenient technology foroperation.

For example, World Patent No. WO2011/121688 discloses a face recognitiondevice which includes an image capture unit, a region detection unit, afeature value extraction unit, a distance estimation unit and a featurevalue update unit. The region detection unit is for detecting apredetermined region in the image captured by the image capture unit.The feature value extraction unit is for extracting a feature value ofthe predetermined region. The distance estimation unit is for estimatinga distance between the image capture unit and the predetermined region.The feature value update unit is for updating the feature valueaccording to the distance. As a result, it is able to realize highrecognition accuracy.

The expression recognition technology is a new technology in developmentthat is able to detect different facial expressions in different moods,such as anger, terror, dislike, happiness, expressionless, sadness,surprise and so on. For example, U.S. Patent Pub. No. 2012169895discloses a method and apparatus for capturing facial expressions thatemploy a recognizer implemented by using a Hierarchical Temporal Memory(MTM) recognizer. The method recognizes a target image so as to obtainan expression status and can be applied to the image capturing devicessuch as a digital camera, Web Cam, IP Cam or Monitor camera.

Accordingly, the present disclosure provides a user recognition andconfirmation device and method, and a central control system forvehicles using the same that may recognize a user ID at first by using aface recognition unit and then confirm the accuracy of the user IDthrough an expression recognition unit. In this manner, the user mayaccomplish the recognition and the confirmation procedures without usingbody motion.

SUMMARY

The present disclosure provides a user recognition and confirmationdevice and method, and a central control system for vehicles using thesame that may recognize the user ID using face recognition technologyand confirm the user ID using expression recognition technology.

The present disclosure further provides a user recognition andconfirmation device and method, and a central control system forvehicles using the same that may be applied to automatic settings ofvarious electronic devices for vehicles.

The present disclosure provides a user recognition and confirmationdevice including an image sensing unit, a face recognition unit, adisplay unit and an expression recognition unit. The image sensing unitis configured to capture a first image frame and a second image frame.The face recognition unit is configured to recognize a user ID accordingto the first image frame. The display unit is configured to show IDinformation of the user ID. The expression recognition unit isconfigured to confirm a user expression according to the second imageframe and output a confirm signal.

The present disclosure further provides a central control system forvehicles including an image sensing unit, a face recognition unit, adisplay unit, an expression recognition unit and a control unit. Theimage sensing unit is configured to capture a first image frame and asecond image frame. The face recognition unit is configured to recognizea user ID according to the first image frame. The display unit isconfigured to show ID information of the user ID. The expressionrecognition unit is configured to confirm a user expression according tothe second image frame and output a confirm signal. The control unit isconfigured to read a driver setting associated with the user IDaccording to the confirm signal.

The present disclosure further provides a user recognition andconfirmation method including the steps of: capturing a first imageframe with an image sensing unit; recognizing a user ID according to thefirst image frame; showing ID information of the user ID with a displayunit; capturing a second image frame with the image sensing unit;confirming a user expression according to the second image frame andgenerating a confirm signal; and controlling at least one electronicdevice according to the confirm signal.

In one aspect, the face recognition unit may be further configured toidentify whether there is a user existing in the first image frameand/or identify a user to be recognized within a plurality of users inthe first image frame.

In one aspect, the user recognition and confirmation device furtherincludes a storage unit configured to save ID information of at leastone user, expression data associated with the user expression andsetting information.

In one aspect, the display unit may show the ID information via screenand/or sound.

In one aspect, the user recognition and confirmation method furtherincludes the steps of: capturing a third image frame with the imagesensing unit; confirming a user expression according to the third imageframe and generating a saving signal; and updating setting informationof the at least one electronic device according to the saving signal.

In the user recognition and confirmation device and method, and thecentral control system for vehicles of the present disclosure, the userID may be automatically confirmed through a user expression not throughbody motion of the user. In addition, when the user desires to updatethe setting information, the updating may also be implemented throughthe expression recognition.

BRIEF DESCRIPTION OF THE DRAWINGS

Other objects, advantages, and novel features of the present disclosurewill become more apparent from the following detailed description whentaken in conjunction with the accompanying drawings.

FIG. 1 shows a schematic block diagram of the user recognition andconfirmation device according to an embodiment of the presentdisclosure.

FIG. 2 shows a schematic diagram of the ID information shown by thedisplay unit of the user recognition and confirmation device accordingto the embodiment of the present disclosure.

FIG. 3 shows a schematic diagram of the central control system forvehicles to which the user recognition and confirmation device accordingto the embodiment of the present disclosure applies.

FIG. 4 shows a flow chart of the user recognition and confirmationmethod according to an embodiment of the present disclosure.

FIGS. 4A to 4B respectively show another flow chart of the userrecognition and confirmation method according to the embodiment of thepresent disclosure.

FIG. 5 shows a schematic block diagram of the user recognition andconfirmation device according to another embodiment of the presentdisclosure.

DETAILED DESCRIPTION OF THE EMBODIMENT

It should be noted that, wherever possible, the same reference numberswill be used throughout the drawings to refer to the same or like parts.

Referring to FIG. 1, it shows a schematic block diagram of the userrecognition and confirmation device according to an embodiment of thepresent disclosure. The user recognition and confirmation device 1includes an image sensing unit 11, a face recognition unit 12, a storageunit 13, a display unit 14, an expression recognition unit 15, a controlunit 16 and an electronic device 17. In this embodiment, the facerecognition unit 12, the expression recognition unit 15 and the controlunit 16 may be implemented by processors capable of processing digitaldata such as a digital signal processor (DSP), and the face recognitionalgorithm and expression recognition algorithm may be previously builtin the digital signal processor.

The user recognition and confirmation device 1 according to theembodiment of the present disclosure is configured to recognize a userID at first and then confirm whether the user ID is correct or notaccording to a user expression so as to accordingly control at least oneelectronic device 17. In one embodiment, the user recognition andconfirmation device 1 may be applied to a central control system forvehicles so as to perform corresponding driver settings according to theuse habit of every driver, wherein the driver settings may include aseat position, a mirror angle, an air conditioner temperature, an audiosetting, a steering wheel height, a light setting and a GPS (globalpositioning system) setting, but not limited to. However, the userrecognition and confirmation device 1 according to the embodiment of thepresent disclosure in not limited to be applied to the central controlsystem for vehicles. It may be applied to any human-machine interfacesystem that may perform the confirmation of the recognized user IDaccording to the facial expression of the user.

The image sensing unit 11 may be a CCD image sensor, a CMOS image senoror the like. The image sensing unit 11 is configured to capture a firstimage frame F1 and a second image frame F2, wherein the image sensingunit 11 may capture images at a fixed capturing frequency. For examplein FIG. 1, the image sensing unit 11 captures images of a user 8 with afield of view (FOV).

The face recognition unit 12 is configured to recognize a user IDaccording to the first image frame F1, wherein after the face feature(s)is calculated using algorithm, the face recognition unit 12 searches thecorresponding user ID in the storage unit 13 and transmits the searcheduser ID to the display unit 14 for displaying. It is appreciated thatthe face recognition unit 12 may start to recognize the user ID onlywhen the first image frame F1 includes a face image.

The display unit 14 may be, for example, a liquid crystal display andhas a display screen 141 and/or a speaker 142 as shown in FIG. 2 forshowing ID information of the user ID. The display unit 14 may also beother types of display. In this manner, the display unit 14 may show theID information via screen and/or sound, wherein the screen may show theuser's picture, drawing or words without any limitation.

The expression recognition unit 15 is configured to confirm a userexpression according to the second image frame F2 and output a confirmsignal Sc, wherein after the expression feature(s) is calculated usingalgorithm, the expression recognition unit 15 searches the expressiondata associated with the user expression in the storage unit 13 forcomparison. In other words, the storage unit 13 may previously save IDinformation of at least one user and the expression data associated withthe user expression. In the embodiment of the present disclosure, saiduser expression is preferably a happiness expression, but not limitedto. The user expression may be set according to different users, e.g.anger, terror, dislike, surprise, sadness or expressionless.

The control unit 16 may control a setting of at least one electronicdevice 17 according to the confirm signal Sc, wherein the setting isassociated with the user ID and may be set by the related user himselfor herself. The setting may be the operational setting of the electronicdevice 17, e.g. operational strength, operational interval and/oroperational directivity, and previously saved in the storage unit 13,wherein the setting may be different according to different electronicdevices 17.

As mentioned above, as the second image frame F2 is for the confirmationof a user ID of the user 8, the image sensing unit 11 may capture thesecond image frame F2 after the display unit 14 has shown the IDinformation. However, the present disclosure is not limited that theimage sensing unit 11 stops its operation in a time interval after theimage sensing unit 11 capturing the first image frame F1 and before thedisplay unit 14 showing the ID information. According to differentapplications the image sensing unit 11 may continuously operate or bedisabled after the first image frame F1 is captured till the expressionrecognition unit 15 recognizes the user expression.

In one embodiment, the image sensing unit 11 starts to operate tocontinuously capture the first image frame F1 after the startingprocedure of the user recognition and confirmation device 1 isaccomplished. Therefore, preferably the face recognition unit 12 may befurther configured to identify whether there is a user 8 existing in thefirst image frame F1. When there is no user image in the first imageframe F1, a new first image frame F1 is captured. When there is a userimage existing in the first image frame F1, the face recognition unit 12starts to recognize a user ID according to the first image frame F1.

In one embodiment, the face recognition unit 12 may be furtherconfigured to identify a user to be recognized within a plurality ofusers in the first image frame F1. For example, when a plurality of userimages are existing in the first image frame F1 simultaneously, the facerecognition unit 12 may select a closest user as the user to berecognized, wherein said closest may be identified according to thesize, contrast and clearness of the user image. In addition, the user tobe recognized may also be identified according to other ways, e.g. theposition of the user image. After the user to be recognized isidentified, the face recognition unit 12 starts to perform the user IDrecognition. In addition, the user recognition unit 12 may also performthe user ID recognition on all of the plurality of users and show therecognized results on the display unit 14 simultaneously. Then, the user8 may confirm the ID information of a user to be recognized via theexpression recognition unit 15.

For example, when the user recognition and confirmation device 1 isapplied to a central control system for vehicles as shown in FIG. 3, theimage sensing unit 11 is preferably disposed opposite to a driver seat.For example, the image sensing unit 11 may be disposed on a rear viewmirror 91 or on a central control panel 92, but not limited to. It mayalso be disposed at any suitable position as long as the driver's faceimage can be captured.

Referring to FIGS. 1 and 3, a central control system for vehiclesemploying the user recognition and confirmation device 1 of the presentdisclosure includes the image sensing unit 11, the face recognition unit12, the storage unit 13, the display unit 14, the expression recognitionunit 15 and the control unit 16. The image sensing unit 11 is configuredto capture the first image frame F1 and the second image frame F2. Theface recognition unit 12 is configured to recognize a user ID accordingto the first image frame F1. The display unit 14 is configured to showID information of the user ID. The expression recognition unit 15 isconfigured to confirm a user expression according to the second imageframe F2 and output a confirm signal Sc. The control unit 16 isconfigured to read the driver setting associated with the user IDaccording to the confirm signal Sc. In this embodiment, in addition tothat the setting is related to the driver setting, e.g. the setting(s)of the electronic device including at least one of a seat, a mirror, anair conditioner, an audio equipment, a steering wheel, a light and a GPSsystem, other operations are similar to those of FIG. 1 andcorresponding descriptions, and thus details thereof are not repeatedherein. Therefore, in this embodiment the storage unit 13 at least savesthe ID information of at least one user, the expression data associatedwith the user expression and the driver setting(s) associated withdifferent electronic devices for vehicles.

In addition, the user recognition and confirmation device 1 of thepresent disclosure may also be able to update the setting information ofat least one electronic device according to the user expression. Acentral control system for vehicles is still used as an example herein,for example the image sensing unit 11 may capture a third image frameand the expression recognition unit 15 may confirm a user expressionaccording to the third image frame and generate a saving signal to thecontrol unit 16. The control unit 16 then saves the current settinginformation of the at least one electronic device into the storage unit13 according to the saving signal so as to update the settinginformation, e.g. updating at least one of a seat position, a mirrorangle, an air conditioner temperature, an audio setting, a steeringwheel height, a light setting and a GPS setting. It is appreciated thatthe step of updating the setting information may be performed when adata update mode is entered, and the data update mode may be enteredaccording to an instruction from the user or directly entered after theexpression recognition unit 15 finishes the confirmation of the IDinformation without any limitation.

Referring to FIG. 4, it shows a flow chart of the user recognition andconfirmation method according to the embodiment of the presentdisclosure, which includes the steps of: capturing a first image framewith an image sensing unit (Step S₂₁); recognizing a user ID accordingto the first image frame (Step S₂₂); showing ID information of the userID with a display unit (Step S₂₃); capturing a second image frame withthe image sensing unit (Step S₂₄); confirming a user expressionaccording to the second image frame and generating a confirm signal(Step S₂₅); controlling at least one electronic device according to theconfirm signal (Step S₂₆); capturing a third image frame with the imagesensing unit (Step S₂₇); confirming a user expression according to thethird image frame and generating a saving signal (Step S₂₈); andupdating setting information of the at least one electronic deviceaccording to the saving signal (Step S₂₉); wherein when the userrecognition and confirmation method of this embodiment is applied to acentral control system for vehicles, said electronic device may be atleast one of a seat, a mirror, an air conditioner, an audio equipment, asteering wheel, a light and a GPS system, but not limited to. Details ofevery step of this embodiment have been described in FIGS. 1 and 3 andcorresponding descriptions and thus they are not repeated herein. FIG. 4describes an embodiment of the performing sequence of every step.

Referring to FIG. 4A, it shows another flow chart of the userrecognition and confirmation method according to the embodiment of thepresent disclosure that may be arranged between the Steps S₂₁ and S₂₂ ofFIG. 4 so as to identify whether there is a user appearing in the firstimage frame F1 (Step S₂₁₁). When there is no user appearing in the firstimage frame F1, the process returns to the Step S₂₁ to capture a newfirst image frame F1. When there is a user appearing in the first imageframe F1, the face recognition unit 12 starts to recognize a user IDaccording to the first image frame F1.

Referring to FIG. 4B, it shows another flow chart of the userrecognition and confirmation method according to the embodiment of thepresent disclosure that may be arranged between the Steps S₂₁ and S₂₂ ofFIG. 4 so as to identify whether there are a plurality of usersappearing in the first image frame F1 (Step S₂₁₂). When there is no userappearing in the first image frame F1, the process returns to the StepS₂₁ to capture a new first image frame F1. When there are a plurality ofusers appearing in the first image frame F1, the face recognition unit12 identifies a user to be recognized within the plurality of users inthe first image frame F1 at first (Step S₂₁₃) and then recognizes a userID of the user to be recognized (Step S₂₂), wherein as mentioned abovethe user to be recognized may be identified according to a user distanceor a user position.

Referring to FIG. 5, it shows a schematic block diagram of the userrecognition and confirmation device according to another embodiment ofthe present disclosure. The difference between FIG. 5 and FIG. 1 is thatin FIG. 5 the second image frame F2 is firstly transmitted to the facerecognition unit 12 and then transmitted to the expression recognitionunit 15, and other parts are similar to FIG. 1. For example in oneembodiment, after the display unit 14 has shown the ID information, theface recognition unit 12 sends the second image frame F2 to theexpression recognition unit 15. In another embodiment, when the facerecognition unit 12 informs the display unit 14 to show the IDinformation, the image frame captured by the image sensing unit 11 (i.e.the second image frame F2) is sent to the expression recognition unit15, and the expression recognition may be performed at the same time thedisplay unit 14 is showing the ID information or after the IDinformation has been shown. In another embodiment, the face recognitionunit 12 and the expression recognition unit 15 may perform respectiverecognition operations using the same image frame; i.e. the first imageframe F1 and the second image frame F2 may be the same image frame.

As mentioned above, the conventional face recognition technology and theexpression recognition technology use different algorithms and havedifferent applications. Therefore, the present disclosure furtherprovides a user recognition and confirmation device (FIG. 1) and a userrecognition and confirmation method (FIG. 4) that may confirm whetherthe user ID is correct or not through the user expression and may beapplied to the central control system for vehicles so as to eliminatethe inconvenience caused by different driver settings associated withdifferent drivers.

Although the disclosure has been explained in relation to its preferredembodiment, it is not used to limit the disclosure. It is to beunderstood that many other possible modifications and variations can bemade by those skilled in the art without departing from the spirit andscope of the disclosure as hereinafter claimed.

What is claimed is:
 1. A user recognition and confirmation device,comprising: an image sensing unit configured to capture a first imageframe and a second image frame; a face recognition unit configured torecognize a user ID according to the first image frame; a display unitconfigured to show ID information of the user ID; and an expressionrecognition unit configured to confirm a user expression according tothe second image frame and output a confirm signal.
 2. The userrecognition and confirmation device as claimed in claim 1, furthercomprising a storage unit configured to save ID information of at leastone user and expression data associated with the user expression.
 3. Theuser recognition and confirmation device as claimed in claim 1, furthercomprising a control unit configured to control a setting of at leastone electronic device according to the confirm signal.
 4. The userrecognition and confirmation device as claimed in claim 3, wherein thesetting is associated with the user ID.
 5. The user recognition andconfirmation device as claimed in claim 3, further comprising a storageunit configured to save the setting.
 6. The user recognition andconfirmation device as claimed in claim 1, wherein the image sensingunit captures the second image frame after the display unit shows the IDinformation.
 7. The user recognition and confirmation device as claimedin claim 1, wherein the face recognition unit is further configured toidentify whether there is a user existing in the first image frame. 8.The user recognition and confirmation device as claimed in claim 1,wherein the face recognition unit is further configured to identify auser to be recognized within a plurality of users in the first imageframe.
 9. A central control system for vehicles, comprising: an imagesensing unit configured to capture a first image frame and a secondimage frame; a face recognition unit configured to recognize a user IDaccording to the first image frame; a display unit configured to show IDinformation of the user ID; an expression recognition unit configured toconfirm a user expression according to the second image frame and outputa confirm signal; and a control unit configured to read a driver settingassociated with the user ID according to the confirm signal.
 10. Thecentral control system as claimed in claim 9, wherein the image sensingunit is disposed opposite to a driver seat.
 11. The central controlsystem as claimed in claim 9, wherein the driver setting comprises atleast one of a seat position, a mirror angle, an air conditionertemperature, an audio setting, a steering wheel height, a light settingand a GPS setting.
 12. The central control system as claimed in claim 9,further comprising a storage unit configured to save ID information ofat least one user, expression data associated with the user expressionand the driver setting.
 13. The central control system as claimed inclaim 9, wherein the image sensing unit captures the second image frameafter the display unit shows the ID information.
 14. The central controlsystem as claimed in claim 9, wherein the face recognition unit isfurther configured to identify whether there is a user existing in thefirst image frame.
 15. The central control system as claimed in claim 9,wherein the face recognition unit is further configured to identify auser to be recognized within a plurality of users in the first imageframe.
 16. The central control system as claimed in claim 9, wherein thedisplay unit shows the ID information through at least one of a screenand a sound.
 17. A user recognition and confirmation method, comprising:capturing a first image frame with an image sensing unit; recognizing auser ID according to the first image frame; showing ID information ofthe user ID with a display unit; capturing a second image frame with theimage sensing unit; confirming a user expression according to the secondimage frame and generating a confirm signal; and controlling at leastone electronic device according to the confirm signal.
 18. The userrecognition and confirmation method as claimed in claim 17, wherein theelectronic device comprises at least one of a seat, a mirror, an airconditioner, an audio equipment, a steering wheel, a light and a GPSsystem.
 19. The user recognition and confirmation method as claimed inclaim 17, further comprising at least one of: identifying whether thereis a user in the first image frame; and identifying a user to berecognized within a plurality of users in the first image frame.
 20. Theuser recognition and confirmation method as claimed in claim 17, furthercomprising: capturing a third image frame with the image sensing unit;confirming a user expression according to the third image frame andgenerating a saving signal; and updating setting information of the atleast one electronic device according to the saving signal.